RTT-D Guidance: 

Implementing Performance Metrics for 
Continuous Improvement that 
Support the Foundational Conditions 
for Personalized Learning 


AMERICAN INSTITUTES FOR RESEARCH 8 

By Jessica Johnson, Kimberly Kendziora, and David Osher 

1000 Thomas Jefferson St., NW 

Washington, DC 20007 

800-356-2735 • 202-403-5000 

www.air.org 

Copyright © 2012 American Institutes for Research. All rights reserved. 


September 2012 



This paper is a companion piece to the "Race to the Top-District Action Brief: Establishing a Foundational Conditions for 
Personalized Learning" by Turnaround for Children. For more information on that paper, please contact Michael Gross, Vice 
President of Partnership Development, Turnaround for Children, mgross@tfcusa.org . 



Table of Contents 


Page 


PURPOSE 1 

CONNECTION TO FOUNDATIONAL CONDITIONS FOR PERSONALIZED LEARNING 1 

AN APPROACH TO CONTINUOUS IMPROVEMENT THAT BUILDS THE CONDITIONS FOR LEARNING, 

PERSONALIZATION, AND ACHIEVEMENT FOR ALL STUDENTS 2 

Critical Elements of the Continuous Improvement Process 2 

Roles in the Continuous Improvement Process 4 

Engaging Stakeholders in Continuous Improvement 6 

APPROACH TO PERFORMANCE METRICS 7 

PROPOSED PERFORMANCE METRICS 8 

Measures of Effective Teachers 8 

Student Surveys as a Measure of Teaching 9 

Measures of Effective Leaders 9 

Measures of Student Academic Engagement and Progress 10 

School Attendance 10 

Grade Retention 11 

Course Performance 11 

Rigorous Coursework 12 

Noncognitive Indicators 14 

Conditions for Learning 14 

Discipline 15 

Social and Emotional Learning (SEL) 16 

Self-Regulation 17 

Physical Health 18 

Mental Health 18 

CONCLUSION 20 

PROPOSED PERFORMANCE METRICS FOR THE THREE FOUNDATIONAL CONDITIONS FOR ESTABLISHING 
PERSONALIZED LEARNING IN RTT-D 21 

REFERENCES 24 

APPENDIX. AIR'S COLLEGE ELIGIBILITY INDEX: A SAMPLE ON-TRACK MEASURE 32 


American Institutes for Research 


RTT-D Guidance— ii 


PURPOSE 


The Race to the Top— District (RTT-D) competition asks districts to personalize education for all 
students in their schools, focusing on classrooms and the relationship between educators and 
students. To reach this bold goal, the competition calls for providing teachers with “the information, 
tools, and supports that enable them to meet the needs of each student and substantially accelerate 
and deepen each student's learning.” Local education agencies must “have the policies, systems, 
infrastructure, capacity, and culture to enable teachers, teacher teams, and school leaders to 
continuously focus on improving individual student achievement and closing achievement gaps.” In 
addition, the RTT-D competition requires that applicants identify 12 to 14 “ambitious yet achievable” 
performance measures, as well as a process for how the measures will provide rigorous, timely, and 
formative information and how the applicant will review and improve the measures over time if they 
are insufficient to support and gauge implementation progress. 

The anchoring system for this work is a continuous improvement process and a set of performance 
measures that drills down to the learner and classroom, focuses on the relationship between 
educators and students, and helps teachers, teacher teams, and school leaders to continuously 
focus on improving individual student achievement and closing achievement gaps. Section E of the 
RTT-D Notice Inviting Applications (NIA) requires that districts have a continuous improvement 
process that includes ongoing communication and engagement and a set of performance measures. 
The proposed continuous improvement process and measures support three conditions necessary 
for successful implementation of personalized learning: (1) effective learning environments, (2) 
effective student support, and (3) organizational efficacy. 


CONNECTION TO FOUNDATIONAL CONDITIONS FOR PERSONALIZED 
LEARNING 


The RTT-D competition presents an important opportunity for leading school districts to design and 
implement continuous improvement processes that support personalized models of teaching and 
learning that can deepen and accelerate college and career readiness for all students. Doing this 
requires metrics that are relevant to the goals of turnaround and personalization, neither of which is 
a simple process. For example, a federally commissioned review of school turnaround (Herman et 
al., 2008) found little rigorous research evidence identifying approaches to school turnaround that 
reliably produce gains in student achievement. This may reflect a lack of research or may suggest 
that there is no one approach that is effective in all circumstances. If the latter is true, locally 
developed processes that may adapt and apply established models may be more appropriate and 
effective. More recent exploratory research suggests that successful turnaround was characterized 
by the interplay of school implementation with district policies and support (Sparks, 2012) and 
another study reported that “all of the case study schools that raised achievement enough to exit 
restructuring used multiple, coordinated strategies, which they revised overtime” (Center for 
Education Policy, 2009, p. 1). 

To be able to successfully implement a personalization of learning strategy successfully, we 
recommend the following foundational strategies as outlined in the companion paper produced by 
Turnaround for Children: 
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• Teacher Practice: Build Highly Effective Classroom Learning Environments. Teachers must be 
trained both to confront recurring challenges and barriers and to effectively build those attributes that 
students must have for deeper learning, and college and career readiness. This means that teachers must 
become proficient in pro-social classroom management and in high-leverage instructional strategies. 
Student Support: Develop a Rigorous Capacity for Student Support. Schools must establish a 
multi-tiered high capacity, high quality student support system that includes school-wide, classroom and 
individual supports for students at all levels of risk and need. This system must include school-wide 
positive discipline, social and emotional learning and classroom-level and individualized student support. 

• Leadership and Management: Establish the Organizational Efficacy to Implement Personalized 
Learning Environments. A multi-disciplinary school leadership team (SLT) must be established to 
develop and execute a school improvement plan (SIP), including putting in place all three foundational 
conditions as the first step in a comprehensive approach to the personalization of learning. The SLT 
should monitor progress and review leading indicators, outcome data, measures of conditions for learning 
and measures that assess quality of implementation. 

There is research to suggest not only that these elements are critical but also that including 
measures related specifically to student support is critical for the success of the reform. In the 
remainder of this report, we describe a continuous improvement strategy and a rigorous application 
of performance measures at the district, school, classroom, and student levels. 


AN APPROACH TO CONTINUOUS IMPROVEMENT THAT BUILDS THE 
CONDITIONS FOR LEARNING, PERSONALIZATION, AND 
ACHIEVEMENT FOR ALL STUDENTS 

In this section, we first describe the critical elements of the continuous improvement process. Next, 
we outline the roles played at the district, school, classroom, and student levels. Finally, we discuss 
how to involve parents and other community stakeholders in the continuous improvement process. 


Critical Elements of the Continuous Improvement Process 

Continuous improvement is a term used to describe the improvement life cycle in which district and 
school staff, along with community stakeholders, (1) collect and analyze data, (2) set measurable 
and achievable goals, (3) plan for improvement using various strategies, resources, and actions, (4) 
implement benchmarks and deliverables, and (5) evaluate progress and modify practice if 
necessary. Although continuous improvement models typically follow this sequential approach over 
the course of a school year, for personalization to be effective, the approach must not be considered 
linear because activities often happen in parallel. It is carried out at multiple levels— district, school, 
classroom, and student. Further, the process includes assigning staff and stakeholders to 
accomplish specific tasks, establishing timelines, and allocating necessary resources. The process is 
“owned” by everyone in the district, schools, and community. This process anchors the 
establishment of personalized learning for all students and must be as follows: 
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• Highly iterative with daily feedback loops at the classroom and student levels. Since 
what happens in the classroom has the greatest effect on student achievement, 
intense focus on understanding data and modifying approaches (instructional and 
others) is critical at this level. This daily focus is important to prevent students from 
experiencing unnecessarily high levels of frustration and to enable students to 
accelerate deeper learning. 

• Fluid and flexible to address that some indicators are likely to lead change, while 
other indicators are more likely to follow improvements on these leading indicators. 

Leading indicators are those individual and environmental factors that, although they 
can change relatively quickly, are predictors of or conditions for longer term change. 
These are strategic outputs (e.g., targeted professional development), individual 
competencies (e.g., improvements in the abilities of students and teachers to 
manage the learning process) individual behaviors (e.g., attendance), and 
improvements in the conditions for learning (e.g., physical and emotional safety). 

• Engaging of a broad group of stakeholders to participate in the development and 
monitoring of district and school plans that focus on 3 to 5 objectives. Although many 
schools typically engage in a school improvement planning process, they do not often 
include a broad stakeholder base. Stakeholder engagement and attention to 
supporting the participation of diverse stakeholders are critical. At the middle and 
secondary levels, we recommend that in addition to administrators, teachers, school 
partners, and parents, students are included in the stakeholder team that co- 
interprets data and generates action plans. In each case, plan development should 
include the resources required for effective implementation, the parties responsible, 
timelines, and proposed measures and benchmarks. Because resources are scarce, 
all efforts should be focused on a set of three to five key objectives for a given school 
year. Stakeholders should be convened twice annually for a “reality check,” where 
they look closely at the data, assess progress, and recommend midcourse 
corrections as needed. The term “reality check" highlights the sometimes difficult 
conversations and decisions that must take place. The school cannot wait until the 
end of the school year to make changes if the data warrant change. Progress reports 
should be provided to the school board quarterly, with accompanying materials (e.g., 
press releases, newsletters) to notify the broader community of the progress. 
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Roles in the Continuous Improvement Process 


DISTRICT 

District and school staff, as well as the stakeholders identified earlier, all play key roles in the 
process. The district is responsible for setting the overall direction and goals for the district. 
Transparency is critical in this process. 


District 

• Conducts annual needs assessment and publicly shares results along with district measures 
and benchmarks 

• Involves parents in the reform and accountability process for student performance 

• Sets annual academic, climate, and social emotional learning targets 

• Convenes school stakeholder committee twice annually to assess progress and to 
recommend changes 

• Establishes systems for identification and support of students with higher levels of need, 
internally and with community support agencies 


SCHOOL 

At the school level, it is critical that a school leadership team (SLT) 1 is structured and given authority 
to carry out the reform. The continuous improvement process and associated reform plan are the 
backbone for this work. For effective reform, the school leadership team that includes teacher 
leaders should be created and convened at a minimum every other week, with clear roles and 
authority. This SLT is responsible for doing the work, monitoring progress, and making midcourse 
corrections as needed. Measures must include leading indicators, outcomes data, measures of the 
conditions for learning, and measures to assess quality (and where relevant, timeliness) of 
implementation, as outlined in the next section. Data should be disaggregated so that monitoring 
and planning can look at results by subgroups. 

In addition to a highly effective school leadership team, a student support team (SST) 2 is needed to 
focus on the provision of services to students with higher levels of academic or social and emotional 
learning (SEL) needs. Implementation of an early warning signs protocol is a critical component for 
this team (see www.betterhighschools.org ). The SST meets weekly to develop and monitor plans for 
high-need students, identified through the early warning signs protocol. (Information on team 
composition and dynamics can be found in chapter 1 of Osher, Dwyer, & Jackson [2004].) 

Finally, providing supports for teachers through access to data, tools, time to plan and use data, and 
professional development are essential. These activities should be identified in the plan and then 
executed with both district and school resources. 


1 To learn more about the SLT, see page 17 of the companion paper to this piece, titled "Race to the Top-District Action Brief: 
Establishing a Foundational Conditions for Personalized Learning'' by Turnaround for Children. 

2 To learn more about the SST, see page 15 of the "Race to the Top-District Action Brief" companion paper to this piece. 
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School 

• Implements the Schoolwide Reform Plan which includes a Learning Environment Plan 

• Establishes and publishes, as a part of the plan, schoolwide measures and benchmarks that 
focus on conditions for learning and teaching as well as measures of academic 
engagement, progress, and achievement 

• Implements an early warning signs protocol for students in need (see 
www.betterhighschools.org) 

• Establishes a school leadership team (SLT) that meets every other week and a student 
support team (SST) that meets weekly 

• Provides data to teachers in a useable format 

• Provides adequate time, tools, and supports for teachers 

• Provide high-quality, on-site, job-embedded professional development supported by 
instructional coaching as needed 


CLASSROOMS 

At the classroom level, teachers are responsible for implementing the strategies and tools they 
acquire through the reform process. From a continuous improvement perspective, they should 
monitor student engagement and learning on a daily basis. Given the demands on teachers, this is 
perhaps the most challenging yet most critical component to continuous improvement. This 
monitoring includes: 

• reviewing predictive data (individual attendance, behavior), 

• formative data on academic performance (formative assessments), and 

• outcome data (achievement scores). 

To make this process continuous, educators should refine practices when warranted, again 
measuring outcomes and looking for places for further refinement. This process must be supported. 
For example, the national Study of Education Data Systems and Decision Making (Means, Padilla, 
DeBarger, & Bakia, 2009) found that despite progress in giving teachers access to student data, 
neither the type of assessments for which data are available nor the time frame of assessment 
activities serves the needs of classroom teachers making decisions on a daily basis. Structures that 
effectively supported data use at the school level included: 

• designated time for teachers to review and discuss data in small groups, 

• assigned support staff to help teachers interpret data, and 

• the adoption of procedures for discussing data. 
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Classroom 

• Implement high-leverage instructional and classroom management strategies that support 
academic and social emotional learning (SEL), the connections between and among 
students and teachers, and personalization 

• Assess student progress daily; meet with colleagues weekly in a professional learning 
community to review student data and to modify practice 

• Participate in formal and informal classroom observations monthly 


STUDENT 

Finally, students and families must engage in the continuous improvement process. Goal setting 
should be a part of student engagement in learning, and teachers should support students in self- 
monitoring progress, which includes awareness of where they are going, how they are doing, and 
where are they going next (Hattie, 2008). 3 This information can, if desired, be part of individual 
growth plans for every student. Families can be engaged through family-driven strategies that employ 
a variety of strategies, such as student-led conferences, curriculum nights, and skill-building courses 
for parents. 


Student 

• Set course-level performance goals based on rubrics for mastery 

• Partner with teacher to develop an individualized plan for college/career readiness 


Engaging Stakeholders in Continuous Improvement 

Stakeholders must be engaged at all levels— district, school, classroom, and student. Groups of 
stakeholders participate in various stages of the continuous improvement process. Identifying them 
up front and determining an engagement strategy with each group is critical. For example, a core set 
of staff, partners, parents, and community members should be identified as a stakeholder group for 
each school and one for the district at large. These groups participate in planning, in midpoint reality 
checks, and in carrying out pieces of the work. The broader school community of all families, staff, 
and students should be engaged at the front end through the needs assessment process and should 
receive frequent, clear communications regarding the objectives and progress toward the plan. 
Stakeholder engagement should include the following: 


3 3 To learn more about involving students in formative assessments and setting their own goals, see page 11 of the "Race to 
the Top-District Action Brief" companion paper to this piece. 
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• An annual needs assessment is an effective mechanism for gathering and reflecting on 
broad stakeholder input. Many districts have some structure in place already to gather 
data to complete a required state-level improvement plan. However, the collection of 
qualitative data and the engagement of a broad stakeholder group are often overlooked 
in this process. Further, the sophistication of analysis of the data needs to include 
disaggregation by subgroups, analysis of trends over time, and triangulation across data 
sources to identify patterns in the current needs and strengths. 

• A multichannel communication plan will increase understanding and generate public 
will and support for the reform. The goals determined by the stakeholder groups should 
be communicated to the entire community through a variety of channels: school 
assemblies, parent meetings, the district website, Facebook, outreach to local religious 
communities, newspaper articles, newsletters, and other community forums. The 
improvement strategy and associated measures and benchmarks should also be widely 
known in the community. 

• District identification of community supports and partnerships reinforces achievement 
and SEL goals. These supports and partnerships include community-based mental 
health providers, local children’s services agencies, afterschool programs, and other 
partnerships that benefit academic or social emotional growth. 


APPROACH TO PERFORMANCE METRICS 

The purpose of performance metrics is to determine success in meeting academic and social- 
emotional learning (SEL) goals. This section of the report connects evidence-based research to 
practices districts can use in their RTT-D applications and provides guidance regarding outcome 
measures. The performance measurement framework as set forth in the RTT-D NIA requires districts 
to establish a set of 12 to 14 performance measures (see section E3 in the RTT-D NIA). To be 
effective, the set of measures must include the following: 
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Key Considerations in Selecting Performance Metrics 

• Cognitive and noncognitive measures. Academic measures generally refer to 
performance on a benchmark or diagnostic test. “Noncognitive measures” may be 
less familiar to educators, but these measures have been firmly established in 
determining school and career outcomes (Borghans, Duckworth, Heckman, & Weel, 
2008; Heckman, & Rubinstein, 2001; Heckman, Stixrud, & Urzua, 2006) and have 
been shown to be related to attendance, discipline, dropout, and achievement 
(Cornell, Gregory, & Huang, in press; Durlak, Weissberg, Dymnicki, Taylor, & 
Schellinger, 2011; Osher, Bear, Sprague, & Doyle, 2010; Osher & Kendziora, 2010). 

• “Quick win” indicators. Quick wins have been cited as important for generating buy-in 
and moving towards full implementation (Herman et al., 2008). Quick wins should be 
successes that strategically move the reform plan forward, (e.g., classroom rules or 
norms in place), rather than truly short-term outcomes (moving “bubble” students to 
proficiency). These ideally occur within the first 30 days of implementation, but they can 
have utility as far as 90 to 180 days into implementation. 

• Leading and progress indicators to predict future outcomes and track ongoing 
process. Leading indicators serve as early signals of progress toward academic 
improvement and achievement. They can help leaders make “more strategic and 
less reactive decisions about services and supports to improve student learning” 
(Foley et. al, 2008, p.2) They include formative and benchmark assessment data on 
the academic side as well as attendance, discipline, and other measures of teacher 
and student engagement on the noncognitive side. Progress indicators are used to 
track ongoing execution and results in terms of implementation as well as 
implementation impact. . 

• Focus on quality measure for ongoing performance excellence. Often districts and 
schools fall into the trap of implementing reforms at a low to moderate level of 
quality, which will be insufficient to change outcomes significantly. For example, 
professional learning communities (PLCs) may be in place, but the conversations 
within them are not rich, and teacher growth is not taking place at the expected 
levels. Quality measures are intended to dig deep into the implementation and 
assess quality. They include observations, surveys, and other specific data about 
what is happening in the school. 


PROPOSED PERFORMANCE METRICS 

The following sections include rationale and suggested metrics for key measures regarding teacher 
and leader performance as well as student engagement and progress, and the “noncognitive” 
measures such as the conditions for learning 

Measures of Effective Teachers 

Rationale. The RTT-D application asks districts to measure teacher and leader effectiveness with a 
specific metric (see the Proposed Performance Metrics table), which asks how many students have 
access to teachers that show gains of one or more years of learning for their students (effective = 1 
year, highly effective = 1.5 or more years). 
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One of the most powerful school variables affecting student achievement is teacher quality (Darling- 
Hammond, 2000). Research on “value-added” assessments of student achievement indicates that 
(1) minority and low-income students are disproportionately taught by under-qualified school 
teachers, and (2) economically disadvantaged students who have effective teachers for consecutive 
years are able to close the achievement gap between disadvantaged and non-disadvantaged 
students. Nye, Konstantopoulos, and Hedges (2004) used Tennessee value-added data to show that 
low-income students were more likely to benefit from instruction by a highly effective teacher 
(defined as a teacher whose average student score gain is in the top 25 percent) than were their 
more advantaged peers. In addition, Sanders and Rivers (1996) reported that the achievement gains 
from having a highly effective teacher could be almost three times as large for African American 
students as for white students, even when comparing students who started with similar achievement 
levels. 

Student Surveys as a Measure of Teaching 

Information from student surveys can provide useful insights into a teacher’s effectiveness as well 
as regarding the conditions for learning, which are described later in this paper. Student feedback 
can be used for non-tested grades and subjects, and when subscales are reliable, the feedback can 
be specific and actionable. In the Measures of Effective Teaching (MET) project, student feedback 
using the Tripod survey (Cambridge Education, 2012) was a better predictor of a teacher’s 
performance than more traditional indicators of success such as whether a teacher had a master’s 
degree or not (Bill & Melinda Gates Foundation, 2010). According to a MET study report, when a 
teacher teaches multiple classes, student perceptions of his or her practice are remarkably 
consistent across different groups of students. The most predictive aspects of student perceptions 
were related to a teacher’s ability to control a classroom and to challenge students with rigorous 
work (Bill & Melinda Gates Foundation, 2010). 


Proposed Teacher Effectiveness Metrics 

Historically, teacher performance has been measured through an annual (or semiannual) 
classroom observation. Although RTT-D demands that student growth be included in 
teacher evaluation, it is critical to note that the review of teacher performance, in a non- 
punitive way, must occur much more frequently than annually or semiannually. 
Consistent with recommendations from the Bill & Melinda Gates Foundations’ Measures 
of Effective Teaching project (2010, 2012), we propose classroom observations (e.g., 
using the CLASS instrument), student surveys (for example, the Tripod survey), and 
instructional coaching and teacher reflection through professional learning communities 
(PLCs) as three possible measures to use on a regular basis. 


Measures of Effective Leaders 

Rationale. For principals, performance is typically measured on an annual basis. Based on the 
school plan, the principal should be held accountable for school metrics on a quarterly basis at 
minimum. At the launch of the new reform, the principal should be held accountable for a “quick 
win”— something that will signal a change in school culture and expectations right from the start. 
Several good examples exist in the turnaround space, such as parent/student outreach, changes to 
physical space, or increase in attendance for the first 10 days of school. 30-, 60-, 90-, and 180-day 
implementation plans and benchmarks are good tools to monitor principal performance. Specific 
indicators must include measures of teacher and student performance schoolwide. 
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Proposed Principal Effectiveness Indicators 

• School rules and procedures are posted and enforced with consistency 
(administrator walk-through) 

• Student learning objectives (SLOs) are defined, shared, and measured weekly 
(teacher report) 

• Teachers are trained on professional learning community (PLC) structure and 
meeting weekly 

• Early warning system (EWS) data collection is in place 

• Student support team (SST) is established 

• School goals are transparent and shared broadly 

• School leadership team (SLT) established, meets regularly with minutes and 
actions 

• Positive behavioral expectations plan in place 

• Classroom observations are a routine part of the administrator’s schedule 


Measures of Student Academic Engagement and Progress 

Student academic indicators measure factors that contribute to academic success. These include 
attendance, grade retention, course promotion, academic progress, and access to rigorous course 
work. In this section, we discuss the rationale for each of these factors as well as describe how they 
can be measured. 

School Attendance 

Rationale. Attendance appears on many district report cards because (1) many schools measure it 
carefully, particularly if they have per-student funding, (2) attendance is often used as a proxy for 
student (and family) engagement in schooling, and (3) attendance is a robust predictor of high 
school dropout (Epstein & Sheldon, 2002; Finn, 1989; Lehr, Sinclair, & Christenson, 2004). 

Research suggests that missing more than 10 percent of instructional time is cause for concern 
(Allensworth & Easton, 2007). This percentage translates to roughly two weeks (9-10 days) of 
school per semester in most schools. 

Attendance is important from the primary grades onward. Research suggests that the combination of 
excused and unexcused absences in elementary school predicts subsequent academic outcomes 
(Chang & Romero, 2008; Connolly & Olsen, 2012). In a sample of students from a large urban 
district, elementary attendance for students who later graduated averaged 94.7 percent, whereas 
those who later dropped out averaged 91.9 percent (Spier, Goosby, & Kendziora, 2005). Eighth- 
grade attendance strongly predicts retention in the ninth grade (Neild & Balfanz, 2006), which in turn 
predicts dropout (Rumberger, 1995). Low attendance during the first 30 days of ninth grade was 
found to be a more powerful predictor of dropout than eighth-grade test scores, academic 
achievement, or age (Jerald, 2006). During the course of high school, attendance continues to be 
significantly associated with the likelihood that a student will graduate (Allensworth & Easton, 2005). 
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Proposed Attendance Metric 

For attendance data to be useful, they should be reported on an individual, not just a 
schoolwide basis. Schools should monitor any student absent more than 10 percent of 
the time and monitor the number of such students by grade. 


Grade Retention 

Rationale. Data recently released by the U.S. Department of Education showed that roughly one 
percent of students in Grades K-8 were retained in grade, with the largest numbers repeating 
kindergarten or first grade (West, 2012). West reported that retention rates were highest among 
traditionally disadvantaged minorities: The respective rates for African American and Latino students 
were 4.2 percent and 2.8 percent, compared with just 1.5 percent for whites. 

Grade retention has been widely regarded as a powerful predictor of dropping out of high school 
(Jimerson, Ferguson, Whipple, Anderson, & Dalton, 2002; Rumberger, 1987, 1995). Repeating a 
grade between first and eighth grades was found to be a significant risk factor for dropping out of 
high school, even after controlling for demographic characteristics that are generally correlated with 
school failure (Janosz, LeBlanc, Boulerice, & Tremblay, 1997; Jimerson, Ferguson, Whipple, 
Anderson, & Dalton, 2002). West (2012) reported results of a rigorous study of grade retention in 
Florida that showed that students who were retained in third grade and provided with intensive 
remediation performed at higher levels than their promoted peers in both reading and mathematics, 
but these effects faded by Grade 7. In addition, students who were overage for their grade in middle 
school for any reason had a higher likelihood of dropping out of high school (Jerald, 2006). 
Exclusionary discipline as well as academic underachievement appear to predict grade retention 
(Fabelo et al., 2011) as does the failure to address issues of credit accumulation for disconnected 
youth (Osher, Amos, & Gonsoulin, 2012). 


Proposed Grade Retention Metric 

Like attendance, grade retention must be monitored on a student-by-student basis. Any 
student who has ever been retained should be flagged to ensure that appropriate 
remediation supports are delivered and later credit accrual is timely. In addition, attention 
should be paid to the credit accumulation of disconnected youth and students with 
chronic health problems. 


Course Performance 

Rationale. Course performance both predicts college completion (via grade point average [GPA]; 
Bowing, Chingas, & McPherson, 2009) and high school dropout. Students who have grades of C or 
lower throughout middle school have increased odds of dropping out of high school, even after 
demographic variables generally associated with school failure are controlled for (Horn & Chen, 
1998). By the end of the first semester of high school, course grades and failure rates are slightly 
better than attendance as predictors of whether students will graduate (Allensworth & Easton, 
2007). Students with a GPA of 2.0 or less at the end of their first year of high school should be 
considered at risk for dropping out. 
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Research from Chicago indicates that students who fail one or more courses in the fall semester of 
their first year of high school are less likely to graduate than students who do not. In Chicago Public 
Schools, 85 percent of students with zero semester course failures in their freshman year graduated 
four years later, but only 70 percent of students with one semester F and only 55 percent of 
students with two semester F’s graduated in four years. Students with three or more semester F’s 
are not likely to graduate high school (Allensworth & Easton, 2007). Research such as this suggests 
that students with one or more F's in any course should be flagged for possible intervention. 



Source: National High School Center, http://www.betterhighschools.org/pubs/ews guide.aso 


Rigorous Coursework 

Rationale. A critical factor determining whether a student enters and completes college is 
participation in a strong academic curriculum in high school (Adelman, 2006). The completion of a 
rigorous high school curriculum was more strongly correlated with completing college than high 
school test scores, GPAs, or race. In addition, an intensive academic curriculum in high school had 
the strongest positive effect for African American and Latino students. Students who enroll in higher 
level, more challenging course work in middle school are more likely to be successful in both high 
school and to attend college (Bridgeman & Wendler, 2005). 

Within the mathematics domain, there is a well-known relationship between completing Algebra I in 
the eighth grade and attending a four-year college (Atanda, 1999; Pelavin & Kane, 1990). A U.S. 
Department of Education report revealed that 83 percent of students who completed both Algebra I 
and geometry began college within two years of high school graduation, compared with only 36 
percent of students who had not taken the two courses. In addition, Algebra I seems to act as an 
“equalizer” across socioeconomic classes. Seventy-one percent of low-income students who 
completed both Algebra I and geometry went on to college, compared with only 27 percent of those 
who had not taken these classes (U.S. Department of Education, 1997). The difference in college 
attendance rates between students of different income levels is greatly reduced when low-income 
students take Algebra I and geometry, although the disparity does not completely disappear. 

American Institutes for Research (AIR) has developed a somewhat complex but predictive College 
Eligibility Index (Bohrnstedt, Kendziora, Brown, Windham, & Dymnicki, 2012; see the Appendix) that 
examines both course-taking and grades obtained in high school English, mathematics, science, 
social studies, and foreign language. This index is based on research showing that students who 
completed four years of mathematics, science, and English in high school were more likely to 
graduate from college (Bowing et al., 2011; Kuh, Kinzie, Buckley, Bridges, & Hayek., 2006). 

Research also suggests that one indicator of educational adequacy is taking high school courses in a 
foreign language, particularly at the AP level (Rothstein, 2000). Similarly, completing higher-level 
mathematics courses such as Algebra II, trigonometry, or calculus were very strong predictors of high 
academic achievement in college (Adelman, 2006). AIR’s index has been piloted with the 2004, 
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2005, and 2006 freshman cohorts in Syracuse, New York, and early results have shown the 
following: 

• Students who were “on track to thrive” (likely to achieve desired “middle class” outcomes for 
college, career, and citizenship) were the most likely to enroll in college (81.9 percent) when 
compared with students who were “on track to graduate” (65.5 percent) or “offtrack” (24.2 
percent). 

• Of students who enrolled in college, a higher percentage of students who were “on track to 
thrive” enrolled in a more selective four-year college (60.0 percent) when compared with 
students who were “on track” (14.2 percent) or “offtrack” (3.6 percent). 


Proposed Course Work Metric: College Eligibility Index: Example for Seniors 


Off Track 

On Track to Graduate 

On Track for a 
Selective College 

English course- 
taking/grades 

Did not take or did not receive 
at least a C in two classes 

Received a C or above 
in two classes 

Received a B or above in 
three classes 

Mathematics course- 
taking/grades 

Did not take or did not receive 
at least a C in two classes 

Received a C or above 
in two classes 

Received a B or above in 
four classes 

Mathematics courses 

Did not take or did not receive 
at least a C in trigonometry, 
precalculus, or calculus 

Received a C or above 
in trigonometry, 
precalculus, or calculus 

Received a B or above in 
trigonometry, 
precalculus, or calculus 

Science course- 
taking/grades 

Did not take or did not receive 
at least a C in three classes 

Received a C or above in 
three classes (at least 
two laboratory classes) 

Received a B or above in 
four classes (at least two 
laboratory classes) 

Social studies course- 
taking/grades 

Did not take or did not receive 
at least a C in two classes 

Received a C or above 
in three classes 
(including one history 
class) 

Received a B or above in 
three classes (including 
one history class) 

Foreign language 
course-taking/grades 

Did not take or did not receive 
at least a C in two years of 
same foreign language 

Received a C or above 
in two to three years of 
the same foreign 
language 

Received a B or above in 
three years of the same 
foreign language 

High school GPA 

Below 2.5 

2.5 to 3.49 

3.5 or higher 


The full College Eligibility Index is presented in the Appendix. 
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Noncognitive Indicators 

There are four sets of noncognitive factors that affect personalization and learning: 

1. Conditions for learning 

2. Schools approach to discipline and how it affects students 

3. Student social-emotional competencies, which, although they have cognitive components, 
are often described as being noncognitive 

4. Student heath— behavioral and physical 

We describe each in the following sections. 


Conditions for Learning 

Rationale. “Conditions for learning” refer to those aspects of school climate that are proximally 
related to learning and development. These conditions can be facilitated by student support, positive 
behavioral approaches, robust curricula, strong and culturally responsive pedagogy, and support for 
social and emotional learning (Osher, Dwyer, Jimerson, & Brown, 2012; Osterman, 2000; Ryan & 
Patrick, 2001; Thuen & Bru, 2009). 

A National Research Council report (Bowman, Donovan, & Burns, 2001) pointed out that “one of the 
most consistent findings in the early childhood literature is that an emotionally warm and positive 
approach in learning situations leads to constructive behavior in children” (p. 52). Pianta and his 
colleagues (2008) have shown that better emotional quality of classroom interactions positively 
predicts growth in reading and mathematics achievement from first through fifth grade. Starting early 
is very important, but across all years of schooling, enhancing social and emotional behaviors can 
have a strong impact on success in school and ultimately in life (Zins, Bloodworth, Weissberg, & 
Walberg, 2004). 

The literature concerning climate and conditions for learning shows an association between school 
climate and academic achievement (e.g., Klem & Connell, 2004; Osher & Kendziora, 2010; Osher, 
Spier, Kendziora, & Cai, 2009). In addition, researchers have shown that regardless of the level of 
school climate, improving school climate is associated with increases in student performance in 
reading, writing, and mathematics (Osher et al., 2009). 
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Proposed Conditions for Learning Metrics 


The Safe and Supportive Schools Technical Assistance Center maintains an online compendium of 
school climate measures on its website . The list includes student, staff, and family surveys that 
can be used as part of a school climate needs assessment. Vetted surveys (and the target 
audience that completes them) are as follows: 

• American Institutes for Research Conditions for Learning Survey (student report) 

• California Healthy Kids Survey (students) 

• California School Climate Survey (staff report) 

• California School Parents Survey (parent report) 

• The Center for Research in Educational Policy School Climate Inventory (staff report) 

• Communities That Care Youth Survey (student report) 

• The Consortium on Chicago School Research Survey of Chicago Public Schools (students 
and staff) 

• Culture of Excellence & Ethics Assessment (students, staff, parents) 

• Effective School Battery (students, teachers) 

• National School Climate Center Comprehensive School Climate Inventory (students, staff, 
parents, community members) 

• Perceived School Experiences Scale (students) 

• Pride Learning Environment Survey (students) 

• Pride Teaching Environment Survey (teachers) 

• Search Institute Creating a Great Place to Learn Survey (students, staff)Secondary 
Classroom Climate Assessment Instrument (students, staff) 

• Secondary School Climate Assessment Instrument (students, staff) 


Discipline 

Rationale. The RTT-D competition requires that districts in which students of color or students with 
disabilities are disproportionately subject to discipline and expulsions must (1) conduct an 
assessment of the root causes of the disproportionality and (2) develop a detailed plan to address 
these root causes. Based on an analysis of data collected by the U.S. Department of Education’s 
Office of Civil Rights, Losen and Gillespie (2012) reported that approximately 839 districts out of 
6,779 in the sample suspended more than 10 percent of their enrolled student body at least once, 
and more than 300 districts suspended more than 25 percent of the African American children 
enrolled. More than 300 districts suspended more than 25 percent of the students with disabilities, 
without regard to race. Patterns of disproportionality are not simple or necessarily what many would 
expect. Skiba, Shure, and Williams (2012) reported that although absolute rates of suspensions are 
highest in poor urban districts, in secondary schools, and among male students, disparities between 
African Americans and whites are highest in suburban districts, elementary schools, and among 
female students. The consequences of punitive, “push-out” discipline policies can be dire: Fabelo et 
al. (2011) found that expulsion from school significantly increased the likelihood that a student 
would repeat a grade, not graduate, or be adjudicated into the juvenile justice system. 
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Proposed Discipline Metric 

All districts should report disaggregated discipline data annually, including the following: 

• Number of students suspended 

• Number of incidents 

• Reasons for out-of school suspensions 

• Days of lost instruction 

From these data, the metric might be the magnitude of the one or two greatest discipline 
disparities. 


Plans to address disparities should build students’ capacities to own and manage their behavior as 
well as teacher and other staff capacity to manage and support behavior (Osher Coggshall, Colombi, 
Woodruff, & Osher, 2012; Osher, Bear, Sprague, & Doyle, 2010). Addressing discipline disparities as 
well as reducing exclusionary approaches require multitiered approaches that are implemented in a 
culturally competent manner (Osher, Cartledge, Oswald, Artiles, Cotinuo, 2004; Osher, Dwyer, 
Jimerson, & Brown, 2012). Data can be used regularly to monitor how these approaches are being 
implemented and the results of their implementation (Osher, VanAcker, et al., 2004). 


Social and Emotional Learning (SEL) 

SEL is a term used to describe the processes through which children and adults acquire and 
effectively apply the knowledge, attitudes, and skills that the Collaborative for Academic, Social, and 
Emotional Learning (CASEL) has identified as 5 key SEL domains: 

• Self-awareness: Recognizing one’s emotions and thoughts and their influence on behavior. 

• Self-management: Regulating one’s emotions, thoughts, and behaviors effectively in different 
situations. 

• Social awareness: Taking the perspective of and empathizing with others. 

• Relationship skills: Establishing and maintaining healthy and rewarding relationships with 
diverse individuals and groups. 

• Responsible decision making: Making constructive and respectful choices about personal 
behavior and social interactions (CASEL, 2012). 

A growing body of research has demonstrated that programs teaching SEL promote positive 
development among children and youth, reduce problem behaviors, and improve academic 
performance, citizenship, and health-related behaviors (CASEL, 2007; Durlak et a I., 2011). SEL can 
be combined productively with positive behavioral approaches, such as PBIS, as well (Bradshaw, 
Bottiani, Osher, & Sugai, in press; Osher, Sprague, et al., 2008). 
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Proposed Social and Emotional Learning Metrics 

There are several good student and teacher-report measures of social and emotional 
competence, which are reviewed in Kendziora, Weissberg, Ji, and Dusenbury (2011). 
Among the best of these are the following: 

• Devereux Student Strengths Assessment (see www.devereux.org) 

• Resilience Module of the California Healthy Kids Survey (see 
http://chks.wested.org/ ) 


Self-Regulation 

Rationale. Self-regulation, which is key to self-management, involves the management of emotions 
and emotion-related behaviors, focusing attention, planning and problem-solving, and delay of 
gratification (Barkley, 1997; Casey et al., 2011; Eisenberg et al., 1997). Students who show positive 
self-regulation are able to demonstrate persistence and attention to tasks, and are able to manage 
their emotions in a way that allows them to benefit from a collaborative classroom environment (e.g., 
handling corrective feedback on their performance from a teacher, successfully resolving 
disagreements with peers). Students who have difficulty managing negative emotions early on are 
especially likely to suffer later behavior problems compared with peers (Eisenberg et al., 1997). 

Research supports a positive relationship between self-regulation and academic outcomes. For 
example, Fantuzzo et al. (2007) found that a strong relationship between mathematics and self- 
regulation. McClelland et al. (2007) found that not only was behavioral regulation positively related 
to early academic achievement, but that growth in behavioral regulation during the course of a 
preschool year predicted growth in children’s school readiness in three areas important to cognitive 
development. 

One special application of self-regulation is the concept of grit (Duckworth, Peterson, Matthews, & 
Kelly, 2007; Duckworth & Quinn, 2009), which is defined as trait-level perseverance and passion for 
long-term goals, but more commonly described as backbone or “stick-to-it-iveness.” In a series of 
studies, grit was positively associated with undergraduate GPA, cadet enrollment in a second year at 
West Point, and better performance in a competitive spelling bee. Grit tends to be associated with 
older age, higher levels of education, and fewer career changes (Duckworth et al., 2007). 


Proposed Self-Regulation Metrics 

Self-regulation is commonly measured using student self-report instruments. 

• An AIR-developed self-regulation scale piloted as part of a school climate survey 
to students in Grades 5-10 in Cleveland found significant correlations with 
reading and mathematics achievement, GPA, and attendance (Ahadi, 2011). 

• Duckworth’s grit scale is available at http://www.sas.upenn.edu/~duckwort/ . 

• The Motivated Strategies for Learning Questionnaire (Pintrich, Smith, Garcia, & 
Mckeachie, 1993) is a good measure of motivation and use of learning strategies. 
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Physical Health 

Rationale. To perform well academically, students must first attend school. It is not surprising that 
students with chronic conditions such as asthma and obesity have poorer attendance than healthy 
students, and improving student health also improves student attendance. There is evidence that 
when schools increase health conditions for students, such as by facilitating student access to 
health services and increasing physical activity, student absences decrease (Basch, 2011). 
Therefore, schools can take steps to improve student attendance by focusing on improving student 
health. 

Wellness is also important. In 2010, the Centers for Disease Control and Prevention (CDC) undertook 
a review of 50 studies that examined the relationship between physical activity time at school and 
academic performance. Just over half of these studies showed that student participation in school- 
based physical activities conferred positive benefits on children’s academic performance, just under 
half showed no effect on academic performance, and only one study showed a negative effect (CDC, 
2010 ). 


Proposed Physical Health Metrics 

• Body mass index (BMI). Sex-specific BMI-for-age percentiles and z-scores are 
calculated from CDC Growth Charts (2010). Overweight is defined as a BMI-for- 
age > CDC 95th percentile; children with a BMI-for-age > 99th percentile are “very 
overweight.” 

• Participation rates in vision, hearing, and dental screenings 

• Percentage of students with chronic conditions such as asthma or diabetes who 
are actively monitored by school nurses. 


Mental Health 

Rationale. For children with mental health needs, schools, not the specialty mental health sector, are 
the primary providers of services (Hoagwood & Erwin, 1997). Schools have not been very successful, 
however, in meeting the needs of children with emotional disturbances. Compared with other 
students with disabilities, students with emotional disturbances are identified later and are more 
likely to be in restrictive placements and drop out of school (U.S. Department of Education, 2011; 
Wagner et a I., 2006). Children with mental health issues that do not involve disruptive behavior, 
such as those with depression or anxiety, are particularly likely to be identified late or not at all. In 
addition, research suggests that that there is a “dual track” to services, in which a disproportionate 
number of children of color as compared to Caucasian children first receive mental health services 
only after they encounter the child welfare or juvenile justice system (Huang, 2007). 

Schools are in a key position to identify mental health problems early and to provide a link to 
appropriate services. Best practice in mental health screening involves a three-stage process. First, 
schools must have a process in place for noticing whether a student may be having an emotional or 
behavioral challenge. In primary settings, teachers can use the Systematic Screening for Behavior 
Disorders (Caldarella, Young, Richardson, Young, & Young, 2008; Walker & Severson 1992). 
Alternatively, a teacher-rating form such as the Vanderbilt Teacher Rating Scale (Wolraich, Feurer, 
Hannah, Pinnock, & Baumgaertel, 1998), Strengths and Difficulties Questionnaire (SDQ; Goodman, 
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1997); or the Behavior Assessment System for Children, Behavioral and Emotional Screening 
System (BASC-2 BESS; Kamphaus & Reynolds, 2007) can be used to screen students based on 
teacher report. 

Following nomination for screening, the family is contacted for consent. The formal screening begins 
with a brief student questionnaire, such as the Columbia Health Screen (for students ages 11-18; 
Husky et al., 2011; Schaffer et al., 2004). Youth who score positive on the screening questionnaire 
or who request to speak with someone should be given a diagnostic clinical interview with a trained 
mental health professional as soon as possible after the screening to determine if further evaluation 
is needed. Only those who score positive on the questionnaire and who are deemed to be at risk by a 
mental health professional receive a referral for further evaluation and intervention planning. 


Proposed Mental Health Metrics 

Paper and pencil measures that can be used in the nomination and screening process 
include the following: 

For systematic teacher nominations: 

• Systematic Screening for Behavior Disorders (Walker & Severson 1992) 

For teacher ratings of student symptoms: 

• Vanderbilt Teacher Rating Scale (Wolraich et al., 1998) 

• Strengths and Difficulties Questionnaire (Goodman, 1997) 

• Behavior Assessment System for Children, Behavioral and Emotional Screening 
System (Kamphaus & Reynolds, 2007) 

For screening student self-report: 

• Columbia Health Screen (Husky et al., 2011; Schaffer et al., 2004) 

Another metric is that appropriate identification and referral processes are in place to 
meet students’ mental health needs. These include the following components: 

• A Student Support Social Worker who provides crisis intervention, on site 
counseling, screening, evaluation and triage 

• A multidisciplinary Student Support Team (SST) for case management and follow 
up of identified students, coordination with child serving agencies and systems 

• Community Child and Family Serving Providers (including Mental Health) to 
provide evaluation and intervention, child welfare services, and juvenile justice 
coordination 


By examining both cognitive and noncognitive data districts, schools, and stakeholders will possess 
critical information needed to assess and potentially modify practice leading to continuous 
improvement for students' school and career outcomes. 
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CONCLUSION 


All young people, particularly students with high needs, require schools with strong foundational 
conditions for learning. These conditions can be addressed by focusing on improvements in three 
important areas: 

1. Teacher Practice - Building highly effective classroom learning environments; 

2. Student Support - Developing a rigorous capacity for student support; 

3. Leadership and Management - Establishing the organizational efficacy to implement 
personalized learning environments. 

Although a relentless focus on instruction matters most (Bryk, Sebring, Allensworth, Luppescu, & 
Easton, 2010; Gamoran, Secada, & Marrett, 2000), teachers and school leaders must establish 
these foundational conditions to personalize learning for all students successfully, and students 
must be ready to engage and benefit from new models for rigorous instruction (Lee & Ready, 2009; 
Lee & Smith, 1999; Sebring, Allensworth, Bryk, Easton, & Luppescu, 2006). 

Approaches to school improvement that use academic achievement as the sole metric to drive 
reform have not proven themselves effective. Because deeper learning and personalization depend 
upon multiple individual and contextual factors, it is important that the continuous improvement 
process address both cognitive and non-cognitive factors. This must be done in a manner that builds 
the capacity of districts, schools, teachers, students, and other stakeholders to contribute to the 
growth of both cognitive and noncognitive attributes that ultimately result in dramatic achievement 
and college and career readiness. Furthermore, this continuous improvement process must be 
supported by processes and structures that collect, analyze, and use data efficiently throughout the 
improvement arc. 

We believe this paper identifies critical metrics and related indicators, along with a continuous 
improvement strategy, to track the establishment of foundational conditions for personalized 
learning. Utilization of these indicators and metrics will provide educators with the ability to meet the 
needs of every student, build upon the all important connection between teachers and students and 
substantially accelerate and deepen student learning toward college and career readiness. 
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PROPOSED PERFORMANCE METRICS FOR THE THREE FOUNDATIONAL CONDITIONS FOR 
ESTABLISHING PERSONALIZED LEARNING IN RTT-D 


Foundational Condition 1— Teacher Practice: Building Highly Effective Classroom Learning Environments 


• Teachers are using high-leverage instructional strategies 

• Positive classroom management is in place 

• All teachers become proficient in building and implementing effective learning environments 


Quick-Win Measures 
First 30 days of reform 

Leading and Progress Indicators 
First 90-360 days of reform 

Quality Measures 
Longer Term Sustainability 

School 

• School rules and 
procedures are posted and 
enforced with consistency 
(administrator walk- 
through) 

• Student learning objectives 
(SLOs) are defined, shared, 
and measured weekly 
(teacher report) 

• Teachers are trained on 
PLC structure and meeting 
weekly 

• Discipline referrals, and disparities in 
referrals by subgroup are reduced 

• Student and teacher attendance trends 
upward and is above 90% 

• Teacher participation in PLCsand 
professional development opportunities 
is consistent, and outliers are 
addressed 

• Student and staff surveys report high levels of engagement 
and positive culture (e.g., Tripod survey; see Bill & Melinda 
Gates Foundation, 2010, Consortium on Chicago School 
Research survey, California School Climate Survey) 

• Observations show appropriate use of high-leverage 
instructional strategies (Framework for Teaching (Danielson 
Group), Classroom Assessment Scoring System (or CLASS, 
Pianta, La Paro, & Hamre, University of Virginia), UTeach 
Teacher Observation Protocol (or UTOP, Marder & Walkington, 
University of Texas-Austin) 

• Teacher evaluations show 100% proficiency on high-leverage 
instructional strategies and classroom management 

Classroom 

• Classroom rules and 
procedures are posted and 
enforced with consistency 
(administrator walk- 
through) 

• Classroom setup is 
conducive to learning 
(administrator walk- 
through) 

• Number/percent of students on track 
based on formative measures of SLOs is 
increasing (teacher, school, or district 
created) 

• Benchmark assessments in English 
language arts and mathematics (MAP, 
Accuity, Diebels) show growth for all 
students 

• Every parent is engaged (direct 
outreach, student-led conferences, 
curriculum events, etc.) 

• Teacher self-assessment data, coaching support and 
feedback, and teacher evaluations are aligned (Coaching 
Tracking Tool, Correlation Analysis) 

• Lesson plans are aligned to the instructional framework and 
include high-leverage instructional strategies (peer, coach, or 
administrator review) 

Student 

• Students feel safe coming 
to school (short survey) 

• Students come to school 
regularly 

• Basic needs are identified 
for all students (breakfast, 
school supplies, health 
care) 

• Students have individual goals for 
learning 

• Students are aware of individual and 
collective growth progress through 
formative and benchmark assessments 
(self-charting, data walls, and data 
assemblies) 

• Student-adult relationships identified 
for at-risk students 

• Basic needs are met for all students (meals, supplies, 
physical/mental health care) 

• Students feel connected to school and motivated to learn 
(survey) 

• Student achievement is increasing (benchmark and standard 
assessments) 

• Students have completed individualized learning plans and are 
making sufficient progress toward plan goals (quarterly plan 
reviews) 
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Foundational Condition 2— Student Support: Developing a Rigorous Capacity for Student Support 


• Discipline and SEL structure is in place 

• Early warning system (EWS) is in place 

• Positive classroom management in place 

• Infrastructure for a high-capacity student support system is in place (student support social worker, linkages to community mental health 
providers, student support team, collaboration with child service agencies) 


Quick-Win Measures 
First 30 days of reform 

Leading and Progress Indicators 
First 90-360 days of reform 

Quality Measures 
Longer Term Sustainability 

School 

• School rules and procedures are 
posted, shared, and enforced 
(administrator walk-through) 

• EWS data collection is in place 

• Student support team is established 

• Initial outreach to key external 
supplemental educational services 
(SES) partnerships has occurred 

• Students are moving through and out 
of interventions at expected rate 

• Number/percent of students failing 
two or more courses (on-track 
indicator) is monitored 

• Students are identified and monitored 
on a weekly basis 

• 100% of students with chronic 
conditions who are actively monitored 
by school nurses, social workers, etc., 
is declining 

• School-based and community 
providers are aligned on referral and 
communication process 

• Number, percent, and mobility of students enrolled 
in differentiated learning programs (i.e. , Read 180 
for tier 2 students) by program and subject show 
student growth 

• Student social-emotional competence is growing 
o Devereux Early Childhood Assessment 
o Devereux Student Strengths Assessment 
(DESSA) and DESSA-Mini (an 8-item quick 
screener) 

o California Healthy Kids Survey, Resilience 
Module 

• Students are receiving appropriate interventions 

Classroom 

• Classroom teachers have access to 
EWS data and know how to contribute 
data 

• Map of intervention options is available 
to teachers so they can access 
resources 

• Instructional time is increasing, and 
time spent managing behavior is 
decreasing (walk-through) 

• Discipline referrals are decreasing 

• Teachers are effectively implementing tier 1 and 
tier 2 strategies in the classroom (observation) 

Student 

• Students know how and who to ask for 
additional supports 

• Students understand consequences of 
their behavior 

• Students are exhibiting self-control 
and monitoring of their behaviors 
(teacher observation) 

• Changes in behavior or achievement 
are noticed by adults (early warning 
protocol) 

• Students’ instructional and social-emotional needs 
are being met, including systematic screening for 
health and mental health issues 

• College Eligibility index shows that the number and 
percentage of students taking sufficiently rigorous 
courses to be on track to enter selective colleges is 
increasing 
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Foundational Condition 3— Leadership & Management: Establishing the Organizational Efficacy to Implement Personalized Learning 
Environments 


• High-functioning school leadership teams (SLTs) 

• Broader stakeholder group participation in planning 

• Programmatic and strategic coherence between district and school 

• Rigorous delivery system for district and school professional development 

• Districtwide infrastructure for the student support system 

• Districtwide infrastructure for data collection, analysis, and reform evaluation 


Quick-Win Measures 
First 30 days of reform 

Leading and Progress Indicators 
First 90-360 days of reform 

Quality Measures 
Longer Term Sustainability 

District 

• Districtwide professional development 
opportunities are communicated 

• District and school reform 
plans are aligned 

• Goals for the district and 
school are broadly 
communicated and known in 
the community 

• Districtwide continuous improvement process promotes 
alignment of activities at the district, school, and 
classroom level 

• Professional learning communities are high functioning 
(observation) 

• Community engagement is strong (high level of 
participation in district and school events) 

• Relationships with community providers are strong, well 
defined, and known throughout the community 

School 

• School goals are transparent and shared 
broadly 

• Roles for school leadership teams are 
documented in writing 

• School leadership team established, 
meets regularly with minutes and actions 

• Positive behavioral expectations plan in 
place 

• Classroom observations are a routine part 
of administrators schedule 

• Systems and structures for 
behavior and academic 
monitoring are in place 
(behavior, SLT, PLC, 
instructional coaching) 

• SLT members are able to 
implement tasks from plan 
(weekly plan monitoring) 

• Targeting coaching and 
professional development are 
in place for SLT as needed 

• Classroom observations show 
positive trends in instructional 
time and student engagement 

• Stakeholder engagement in 
planning is the norm 

• School planning process is aligned to the district but also 
reflects and addresses teacher needs (needs 
assessment) 

• Plans contain multiple measures, quick strategic wins, 
leading indicators , quality of implementation, and so on, 
and are monitored and reported on regularly 
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APPENDIX. 

AIR’S COLLEGE ELIGIBILITY INDEX: A SAMPLE ON-TRACK MEASURE 


The College Eligibility Index for high school students is based on the application requirements for 
selective, four-year colleges. The index is composed of seven indicators: performance in English, 
mathematics, science, social studies, and foreign language courses, mathematics course-taking, and 
high school grade point average. The specific calculations for each indictor are detailed in the 
following sections. The College Eligibility Index overall is scored as follows: 


• “Off track” if a student has two or more of the seven indicators scored in the “off track” 
range, 

• “On track” if no more than one indicator is “off track” and the rest are all “on track” or better, 
and 

• “On track to thrive” if a student has at least five indicators in the “on track to thrive” range 
and the rest are “on track.” 
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Specific Math Courses Taken 


Indicators for Freshmen 




(2) Did not take or did 

(2) received below a 

(3) received a C or 

(4) received a B or 

not pass Algebra 1 

C in Algebra 1 

above in Algebra 1 

above in Algebra 1 


On track to thrive 

4 



On track 

3 



Off track 

1 or 2 


Indicators for Sophomores 



(1) Did not take or did 

(2) received below a 

(3) received a C or 

(4) received a B or 

not pass geometry 

C in geometry 

above in geometry 

above in geometry 


On track to thrive 

4 



On track 

3 



Off track 

1 or 2 


Indicators for Juniors 




(1) Did not take or did 

(2) received below a 

(3) received a C or 

(4) received a B or 

not pass Algebra 2 

C in Algebra II 

above in Algebra II 

above in Algebra II 


On track to thrive 

4 



On track 

3 



Off track 

1 or 2 


Indicators for Seniors 




(1) Did not take or did 

(2) received below a 

(3) received a C or 

(4) received a B or 

not pass trigonometry* 

C in trigonometry 

above in trigonometry 

above in trigonometry 


* Seniors could have also taken alternative mathematics classes (i.e., precalculus or calculus) to 
which these same standards would apply. 


On track to thrive 

4 

On track 

3 

Off track 

1 or 2 
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Performance in Science Courses 
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Performance in Social Studies 
Courses 


Indicators for Freshmen 

(1) Did not take or did 
not pass 1 class 

(2) received below a 
C in 1 class 

(3) received a C or 
above in 1 class 

(4) received a B or 
above in 1 class 


On track to thrive 

4 



On track 

lor 3 



Off track 

2 



Indicators for Sophomores 



(1) Did not take or did 

(2) received below a 

(3) received a C or 

(4) received a B or 

not pass 1 class 

C in 1 or more 

above in 1 or more 

above in 1 or more 


classes 

classes 

classes 


On track to thrive 

4 



On track 

lor 3 



Off track 

2 



Indicators for Juniors 

(1) Did not take or did 

(2) received below a 

(3) received a C or 

(4) received a B or 

not pass 1 class 

C in 1 or more 

above in 2 classes 

above in 2 classes 


classes 




On track to thrive 

4 



On track 

3 



Off track 

1 or 2 



Indicators for Seniors 

(1) Did not take or did 
not pass 2 classes 

(2) received below a 
C in 2 classes 

(3) received a C or 
above in 3 classes 
(including 1 history 
class) 

(4) received a B or 
above in 3 classes 
(including 1 history 
class) 


On track to thrive 

4 



On track 

3 



Off track 

1 or 2 
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